103 research outputs found
Adaptive PD Control using Deep Reinforcement Learning for Local-Remote Teleoperation with Stochastic Time Delays
Local-remote systems allow robots to execute complex tasks in hazardous
environments such as space and nuclear power stations. However, establishing
accurate positional mapping between local and remote devices can be difficult
due to time delays that can compromise system performance and stability.
Enhancing the synchronicity and stability of local-remote systems is vital for
enabling robots to interact with environments at greater distances and under
highly challenging network conditions, including time delays. We introduce an
adaptive control method employing reinforcement learning to tackle the
time-delayed control problem. By adjusting controller parameters in real-time,
this adaptive controller compensates for stochastic delays and improves
synchronicity between local and remote robotic manipulators. To improve the
adaptive PD controller's performance, we devise a model-based reinforcement
learning approach that effectively incorporates multi-step delays into the
learning framework. Utilizing this proposed technique, the local-remote
system's performance is stabilized for stochastic communication time-delays of
up to 290ms. Our results demonstrate that the suggested model-based
reinforcement learning method surpasses the Soft-Actor Critic and augmented
state Soft-Actor Critic techniques. Access the code at:
https://github.com/CAV-Research-Lab/Predictive-Model-Delay-CorrectionComment: 7 pages + 1 references, 4 figure
Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies
Current methods of imitation learning (IL), primarily based on deep neural
networks, offer efficient means for obtaining driving policies from real-world
data but suffer from significant limitations in interpretability and
generalizability. These shortcomings are particularly concerning in
safety-critical applications like autonomous driving. In this paper, we address
these limitations by introducing Symbolic Imitation Learning (SIL), a
groundbreaking method that employs Inductive Logic Programming (ILP) to learn
driving policies which are transparent, explainable and generalisable from
available datasets. Utilizing the real-world highD dataset, we subject our
method to a rigorous comparative analysis against prevailing
neural-network-based IL methods. Our results demonstrate that SIL not only
enhances the interpretability of driving policies but also significantly
improves their applicability across varied driving situations. Hence, this work
offers a novel pathway to more reliable and safer autonomous driving systems,
underscoring the potential of integrating ILP into the domain of IL.Comment: 12 pages, 2 figures, 2 table
Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies
Deep learning has become an increasingly common technique for various control
problems, such as robotic arm manipulation, robot navigation, and autonomous
vehicles. However, the downside of using deep neural networks to learn control
policies is their opaque nature and the difficulties of validating their
safety. As the networks used to obtain state-of-the-art results become
increasingly deep and complex, the rules they have learned and how they operate
become more challenging to understand. This presents an issue, since in
safety-critical applications the safety of the control policy must be ensured
to a high confidence level. In this paper, we propose an automated black box
testing framework based on adversarial reinforcement learning. The technique
uses an adversarial agent, whose goal is to degrade the performance of the
target model under test. We test the approach on an autonomous vehicle problem,
by training an adversarial reinforcement learning agent, which aims to cause a
deep neural network-driven autonomous vehicle to collide. Two neural networks
trained for autonomous driving are compared, and the results from the testing
are used to compare the robustness of their learned control policies. We show
that the proposed framework is able to find weaknesses in both control policies
that were not evident during online testing and therefore, demonstrate a
significant benefit over manual testing methods.Comment: 2020 IEEE International Conference on Robotics and Automation (ICRA
Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach
The dynamic nature of driving environments and the presence of diverse road
users pose significant challenges for decision-making in autonomous driving.
Deep reinforcement learning (DRL) has emerged as a popular approach to tackle
this problem. However, the application of existing DRL solutions is mainly
confined to simulated environments due to safety concerns, impeding their
deployment in real-world. To overcome this limitation, this paper introduces a
novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logics
(DRLSL) that combines the strengths of DRL (learning from experience) and
symbolic first-order logics knowledge-driven reasoning) to enable safe learning
in real-time interactions of autonomous driving within real environments. This
innovative approach provides a means to learn autonomous driving policies by
actively engaging with the physical environment while ensuring safety. We have
implemented the DRLSL framework in autonomous driving using the highD dataset
and demonstrated that our method successfully avoids unsafe actions during both
the training and testing phases. Furthermore, our results indicate that DRLSL
achieves faster convergence during training and exhibits better
generalizability to new driving scenarios compared to traditional DRL methods.Comment: 15 pages, 9 figures, 1 table, 1 algorithm. Under review as a journal
paper at IEEE transactions on Intelligent Transportation System
Weakly Supervised Reinforcement Learning for Autonomous Highway Driving via Virtual Safety Cages
The use of neural networks and reinforcement learning has become increasingly
popular in autonomous vehicle control. However, the opaqueness of the resulting
control policies presents a significant barrier to deploying neural
network-based control in autonomous vehicles. In this paper, we present a
reinforcement learning based approach to autonomous vehicle longitudinal
control, where the rule-based safety cages provide enhanced safety for the
vehicle as well as weak supervision to the reinforcement learning agent. By
guiding the agent to meaningful states and actions, this weak supervision
improves the convergence during training and enhances the safety of the final
trained policy. This rule-based supervisory controller has the further
advantage of being fully interpretable, thereby enabling traditional validation
and verification approaches to ensure the safety of the vehicle. We compare
models with and without safety cages, as well as models with optimal and
constrained model parameters, and show that the weak supervision consistently
improves the safety of exploration, speed of convergence, and model
performance. Additionally, we show that when the model parameters are
constrained or sub-optimal, the safety cages can enable a model to learn a safe
driving policy even when the model could not be trained to drive through
reinforcement learning alone.Comment: Published in Sensor
- …